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DECISION ON APPEAL 



This is a decision on appeal from the final rejection of claims 1-20. 
The invention is directed to a system for repositioning and resizing the picture- 
in-picture feature of a display based on speech and gesture control. 
Representative independent claim 1 is reproduced as follows: 



1 



Appeal No. 2005-2290 
Application No. 09/896,199 

1 . A video display device comprising: 

a display configured to display a primary image and a 
picture-in-picture image (PIP) overlaying the primary image; and 

a processor operatively coupled to the display and configured to 
receive a first video data stream for the primary image, to receive a 
second video data stream for the PIP, and to change a PIP display 
characteristic in response to a received audio command and a related 
gesture from a user. 

The examiner relies on the following references: 

Inagaki 5,999,214 Dec. 07, 1999 

Pavlovic et al., "Integration of AudioA/isual Information for Use in 
Human-Computer Intelligent Interaction", Image Processing, 1997 
Proceedings IEEE, pages 121-124. 

Claims 1-20 stand rejected under 35 U.S.C. §103 as unpatentable over Inagaki 
in view of Pavlovic 

Reference is made to the brief and answer for the respective positions of 
appellant and the examiner. 

OPINION 

In rejecting claims under 35 U.S.C. §103, it is incumbent upon the 
examiner to establish a factual basis to support the legal conclusion of obviousness. 
See In re Fine , 837 F.2d 1071, 1073, 5 USPQ2d 1596, 1598 (Fed. Cir. 1988). In so 
doing, the examiner is expected to make the factual determinations set forth in Graham 
v, John Deere Co. , 383 U.S. 1 , 17, 148 USPQ 459, 467 (1966), and to provide a reason 
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why one having ordinary skill in the pertinent art would have been led to modify the 
prior art or to combine prior art references to arrive at the claimed invention. Such 
reason must stem from some teachings, suggestions or implications in the prior art as a 
whole or knowledge generally available to one having ordinary skill in the art. Uniroval. 
Inc. v. Rudkin-Wilev Corp. . 837 F.2d 1044, 1051, 5 USPQ2d 1434, 1438 (Fed. Cir.), 
cert, denied . 488 U.S. 825 (1988); Ashland Oil. Inc. v. Delta Resins & Refractories. Inc. 
, 776 F.2d 281, 293, 227 USPQ 657, 664 (Fed. Cir. 1985), cert, denied . 475 U.S. 1017 
(1986); ACS Hosp. Svs.. Inc. v. Montefiore Hosp. . 732 F.2d 1572, 1577, 221 USPQ 
929, 933 (Fed. Cir. 1984). These showings by the examiner are an essential part of 
complying with the burden of presenting a prima facie case of obviousness. Note In re 
Oetiker . 977 F.2d 1443, 1445, 24 USPQ2d 1443, 1444 (Fed. Cir. 1992). If that burden 
is met, the burden then shifts to the applicant to overcome the prima facie case with 
argument and/or evidence. Obviousness is then determined on the basis of the 
evidence as a whole and the relative persuasiveness of the arguments. See ]g\; In re 
Hedges . 783 F.2d 1038, 1040, 228 USPQ 685, 687 (Fed. Cir. 1986); In re Piasecki . 
745 F.2d 1468, 1472, 223 USPQ 785, 788 (Fed. Cir. 1984); and In re Rinehart . 531 
F.2d 1048, 1051, 189 USPQ 143, 146-147 (CCPA 1976). Only those arguments 
actually made by appellant have been considered in this decision. Arguments which 
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appellant could have made but chose not to make in the brief have not been 
considered and are deemed to be waived [see 37 CFR §41 .67(c)(1 )(vii)]. 

With regard to independent claim 1, it is the examiner's position that Inagaki 
discloses the claimed subject matter but for the limitation "to change a PIP display 
characteristic in response to a received audio command and a related gesture from a 
user." Instead, the examiner points out, Inagaki detects and responds to any of the 
many sounds or audio indications in the form of a unique voice of a specific speaking 
attendee with the same command which moves the camera and highlights the PIP of 
the speaking attendee, but response is not dependent on a related gesture from a user. 

The examiner turns to Pavlovic for the concept of a system utilizing a 
combination of audio commands and a related gesture "from a user as a means of 
controlling a graphical object on display which is analysis [analogous?] to where 
Inagaki controlled a specific graphical object such as a PIP on a display." The 
examiner referring to page 123 of Pavlovic, in the EXPERIMENTAL RESULTS section, 
(answer, page 4). 

The examiner concludes that it would have been obvious to use a "received 
audio command and a related gesture from a user," as per Pavlovic, in the system of 
Inagaki because of Pavlovic's own reasoning, viz., "Psychological studies, for example, 
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show that people prefer to use hand gestures in combination with speech in a virtual 
environment , since they allow the user to interact without special training or special 
apparatus" (page 121 of Pavlovic-the examiner's emphasis, at page 4 of the answer). 

Appellant responds by arguing that Inagaki's system merely detects the 
presence of speech (specifically, the presence of speech of a speaking attendee, and 
then highlights the PIP of the speaking attendee). Inagaki does not however, contends 
appellant, disclose or suggest the changing of a PIP display characteristic in response 
to a received audio command and a related gesture from a user. Since Inagaki does 
not detect any content of the speech of the speaking attendee, appellant contends that 
it cannot be said that Inagaki determines if a command is being spoken. Accordingly, 
argues appellant, there would have been no motivation for combining Inagaki with the 
gestures taught by Pavlovic. 

We have considered the evidence before us, including the arguments of 
appellant and the examiner, and we conclude therefrom that the examiner has 
established a prima facie case of obviousness which has not been overcome by 
appellant. Accordingly, we will sustain the rejection of claims 1-20 under 35 U.S.C. 
§103. 

Inagaki clearly teaches the movement of a camera to a different conference 
attendee, dependent on the attendee's voice (see column 11, line 65, through column 
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12, line 25). Since a different attendee will appear larger on the display screen, clearly 
there is a change in a "PIP display characteristic," in response to an audio "command," 
or "indication." We note, as did the examiner, that only independent claim 1 calls for an 
audio "command." The other independent claims call only for an audio "indication." 
Moreover, as the examiner noted, the original disclosure appeared directed only to an 
"indication." Therefore, we believe the examiner has rightly, and reasonably, 
interpreted the claimed "command" to be any type of audio "indication," 

The voice direction detection unit of Inagaki may be said to detect an audio 
"indication," or "command." But, in any event, Pavlovic clearly teaches a "command," 
by any definition, which, along with a gesture, is used to perform some action. See, for 
example, page 123, right-hand column, of Pavlovic. When Pavlovic's teaching is 
viewed in light of Inagaki 1 disclosure of controlling a PIP display characteristic 
automatically, through voice, we conclude, as did the examiner, that the skilled artisan 
would have been led to employ Pavlovic's dual, i.e., speech/gesture control system to 
control the panning of Inagaki's camera. 

We are unpersuaded by appellant's argument anent not sensing the "content of 
the speech" in Inagaki, since not only is such a limitation not a part of the instant claim 
language, but Pavlovic clearly discloses sensing the content of the speech used to 
control the action, as at page 123, where the user points to an object, while 
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simultaneously commanding, "move left" in order to effect such movement of the object 
pointed to. 

We are also unpersuaded by appellant's argument anent no motivation to 
combine the references since, as indicated supra , the skilled artisan had more than 
sufficient motivation to make the combination. 

Specifically with regard to claim 2, appellant argues that Inagaki merely shows 
the detection of a voice of a speaking attendee and "highlights" the PIP of the speaking 
attendee to distinguish that attendee from the other attendees. But, argues appellant, 
highlighting is neither changing the position of the PIP nor the display size of the PIP, 
as required by instant claim 2. 

The examiner contends that the limitations of claim 2 are met because Figure 8a 
of Inagaki "illustrates the concept of a relationship between who is speaking and the 
position of which PIP to be highlighted which further changes when speaker changes 
which is clearly illustrated in figure 8B so therefore it reads on this broad language" 
(sic, answer-page 4). 

We agree with appellant that a different speaker may be highlighted in Inagaki, 
but the position and size of the speaker, i.e., the PIP display, appears to remain the 
same. There is no indication in Inagaki that either the position or size, or both, of the 
PIP display is changed. 



7 



Appeal No. 2005-2290 
Application No. 09/896,199 

However, Pavlovic does teach the movement of an object by a combination of 
spoken command and hand gesture, so the display of an object does change position. 
In making the combination of Inagaki with Pavlovic, for reasons enunciated supra , it 
would have been obvious to the artisan that the teaching, by Pavlovic, of moving the 
position of an object by spoken command/hand gesture would be applicable to the 
movement of any object on a display screen, including the PIP display of Inagaki. 

Accordingly, we will sustain the rejection of claim 2 under 35 U.S.C. §103. 

Specifically with regard to independent claim 11, appellant argues that neither of 
the cited references teaches or suggests "determining whether the received audio 
indication is one of a plurality of expected audio indications." 

We agree with the examiner that Figure 7 of Pavlovic clearly shows a plurality of 
expected hand gestures as well as a plurality of expected speech commands. In fact, 
in order to make the correct movement of an object, the system of Pavlovic must 
determine whether it has received an expected command, such as "move left," which, in 
combination with the correct hand gesture, will effect that movement. 

Accordingly, we will sustain the rejection of claim 1 1 under 35 U.S.C. §103. 

Finally, appellant argues that the limitations of claims 5 and 6 are not taught or 
suggested by the cited references. In particular, appellant argues that since Pavlovic 
discloses issuing a spoken command and a gesture simultaneously, it cannot meet the 
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claim language requiring the analysis of image information after the audio indication is 
received to identify the change in the PIP display characteristic that is expressed by the 
received gesture (see page 17 of the brief). 

We agree with the examiner that the broad claimed subject matter language 
would have been obvious over the disclosure of Pavlovic, e.g., in Figure 5, of receiving 
audio information and gesture information and, thereafter, computing the likelihood of 
such a combination. Thus, the analysis does occur after receipt of the data to be 
analyzed, as it must. If appellant is contending that first the spoken command must be 
received and then, at a later time, the gesture information is received, and only then, at 
a still later time, an analysis is made from this data, we find no patentable distinction, 
and appellant has pointed to none, in receiving bits of data needed for an analysis at 
different times. If a processor requires data A and data B in order to make an analysis, 
it hardly matters, in our view, whether A and B are received simultaneously or at 
different times. The processor still cannot make the analysis until it has all the data 
necessary. Now, we suppose there may be some circumstances where some 
advantage may be gained by receiving data at different times, but, clearly, the instant 
specification and/or claims do not shed any light on what that might be. Thus, as 
disclosed and claimed, we find that the skilled artisan would have discerned no 
unobvious difference between receiving audio and visual data at different times or 
simultaneously. 
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Accordingly, we will sustain the rejection of claims 5 and 6 under 35 U.S.C. 

§103. 

Since appellant makes no separate arguments as to the merits of any other 
claim, we will sustain the rejection of claims 1-20 under 35 U.S.C. §103. 
The examiner's decision is affirmed. 

No time period for taking any subsequent action in connection with this appeal 
may be extended under 37 CFR § 1 .1 36(a) (1 ) (iv). 



AFFIRMED 
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